From Rational Number Reconstruction to Set Reconciliation and File Synchronization

This work revisits set reconciliation, the problem of synchronizing two multisets of xedsize values while minimizing transmission complexity. We propose a new number-theoretic reconciliation protocol called Divide and Factor (D&F) that achieves optimal asymptotic transmission complexity as do previously known alternative algorithms. We analyze the computational complexities of various D&F variants, study the problem of synchronizing sets of variable-size les using hash functions and apply D&F to synchronize le hierarchies taking le locations into account. We describe btrsync, our open-source D&F implementation, and benchmark it against the popular software rsync. It appears that btrsync transmits much less data than rsync, at the expense of a relatively modest computational overhead.